Skip to content

Add PTODSL A5 DSL ST coverage#886

Merged
zhangstevenunity merged 3 commits into
hw-native-sys:mainfrom
jimmychou0:codex/ptodsl-a5-dsl-st-validation
Jul 6, 2026
Merged

Add PTODSL A5 DSL ST coverage#886
zhangstevenunity merged 3 commits into
hw-native-sys:mainfrom
jimmychou0:codex/ptodsl-a5-dsl-st-validation

Conversation

@jimmychou0

@jimmychou0 jimmychou0 commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

Abstract

This PR adds the first PTODSL-authored A5 DSL ST coverage and updates the PTODSL simulator CI path so the new cases are actually built and run.

The branch was rebuilt after dropping the earlier broad backend workaround. The remaining backend change is intentionally narrow: FoldTileBufIntrinsics now performs fixpoint cleanup of dead view chains exposed after tile intrinsic folding. It deletes only use-empty bridge casts, memref view ops, pto.make_tensor_view / pto.partition_view, and dead tile allocations; it does not rerun full PTOViewToMemref, does not broaden ExpandTileOp, and does not change live view lowering semantics.

Problem scenarios covered:

  • PTODSL A5 vector tile-op coverage: tadd validates a basic tload + tadd + tstore path outside the old tilelang_st harness.
  • PTODSL data movement coverage: tload_store validates GM view construction, tload, tstore, and layout variants.
  • PTODSL broadcast/reduction-style tile coverage: tcolexpand and tcolsum cover non-trivial tile shapes, valid rows/cols, and tile-op expansion/runtime behavior.
  • PTODSL cube coverage: tmatmul validates a cube tile matmul path, while the existing cube_matrix_pipeline.py and gemv_mx_pipeline.py remain part of the simulator suite.
  • Explicit native build policy: mode="explicit" kernels need to compile through PTOAS level3 and should not implicitly enable sync insertion.
  • Native build cache correctness: changing effective compile policy, such as PTO level or insert-sync behavior, must invalidate cached .so artifacts.
  • PTODSL helper/container shape: explicit same-kind @pto.simd / @pto.cube should not create redundant section wrappers; explicit kind mismatches should fail early with a clear diagnostic.
  • Dead PTODSL view chains: after tile-op expansion/folding, dead make_tensor_view / partition_view chains can otherwise leave high-level or memref view ops that later VPTO emission validation rejects.

Implementation changes:

  • Add test/dsl-st/npu_a5 cases for tadd, tload_store, tcolexpand, tcolsum, and tmatmul.
  • Update existing DSL ST cases for the validated simulator flow, including pointer-form vector load/store operands in predicate_pack.py and vmulscvt.py to avoid level3 live memref subviews.
  • Track whether kernel_kind was explicitly authored in @pto.jit while preserving the historical default effective kind of vector when omitted.
  • Lower same-kind explicit subkernel scopes using function/kernel-kind metadata, and diagnose explicit kernel-kind/subkernel-kind mismatches.
  • Map PTODSL native mode="explicit" to ptoas --pto-level=level3; keep explicit mode from implicitly enabling insert-sync.
  • Add compile-configuration hashing to the PTODSL native build cache manifest.
  • Add fixpoint dead-chain cleanup to FoldTileBufIntrinsics and a focused VPTO lit regression.
  • Handle single-child VPTO backend container compile units in tools/ptoas/driver.cpp for non-debug output paths.
  • Update PTODSL simulator CI to use an existing torch / torch_npu runtime instead of installing them each run, isolate PTODSL build artifacts, and ensure test/dsl-st/npu_a5 is covered.

Validation

Validated on the 144 simulator environment under /home/zhoujiaming/ptoas-sim-ci/pr886-cleanup using the LLVM21 VPTO build and CANN simulator:

ninja -C build-sim ptoas PTOPythonModules
/home/zhoujiaming/ptoas-sim-ci/venv-llvm21-build/bin/python \
  /home/zhoujiaming/ptoas-sim-ci/llvm-project-vpto21/build-assert/bin/llvm-lit -sv build-sim/test/lit
/home/zhoujiaming/ptoas-sim-ci/PTOAS-ptodsl-main/.venv/bin/python ptodsl/tests/test_jit_compile.py
scripts/sim_dsl.sh --soc-version Ascend950PR_9599 test/dsl-st

Results:

  • llvm-lit: 764/764 passed.
  • ptodsl/tests/test_jit_compile.py: passed.
  • scripts/sim_dsl.sh --soc-version Ascend950PR_9599 test/dsl-st: all cases passed, including cube_matrix_pipeline, gemv_mx_pipeline, predicate_pack, simt_gm_memory_core, vmulscvt, and the new npu_a5 directory coverage.

Local checks:

git diff --check HEAD~1..HEAD

Also checked that the final diff no longer touches the old broad-workaround files such as ExpandTileOp.cpp, PTOInstantiateAndInlineOpLib.cpp, Passes.td, tools/ptoas/ptoas.cpp, VPTOOps.td, VPTO.cpp, or VPTOPtrNormalize.cpp.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces native tensor_view and partition_tensor_view folding support in the FoldTileBufIntrinsics pass, updates ExpandTileOp to include view shape and strides in the specialization key, and adds a pto_level parameter to @pto.jit to forward build-level overrides to ptoas. Additionally, VPTOSplitCVModule is updated to normalize sections in-place for pre-annotated modules. Feedback on the changes highlights a concurrency violation in FoldTileBufIntrinsics where a FuncOp pass queries the parent module's symbol table, a limitation in traceViewChain that fails on nested partitions, and an inefficient cleanup loop that should be optimized using a worklist-based dead code elimination approach.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread lib/PTO/Transforms/FoldTileBufIntrinsics.cpp Outdated
Comment thread lib/PTO/Transforms/FoldTileBufIntrinsics.cpp Outdated
Comment thread lib/PTO/Transforms/FoldTileBufIntrinsics.cpp Outdated
@reedhecre

reedhecre commented Jun 30, 2026

Copy link
Copy Markdown

Codex Review

该评论由 review 机器人自动更新。

  • PR: Add PTODSL A5 DSL ST coverage #886 Add PTODSL A5 DSL ST coverage
  • Author: jimmychou0
  • Base/Head: main / codex/ptodsl-a5-dsl-st-validation
  • Head SHA: c35bac0886cb
  • Trigger: PR 有新提交
  • Generated At: 2026-07-06T08:20:43Z
  • Previous Head SHA: 445f437938ca
  • Status: failed at codex-review (exit=1)

Summary

Review failed at stage codex-review: exit=1

Findings

未生成结构化 findings,因为 review 过程提前失败。

Log Tail

 ptodsl/ptodsl/_tracing/module_builder.py           |   1 +
 ptodsl/ptodsl/_tracing/session.py                  |  47 +-
 ptodsl/tests/test_jit_compile.py                   |  98 ++++-
 ptodsl/tests/test_runtime_toolchain.py             |  51 +++
 scripts/ptoas_env.sh                               |   9 +-
 scripts/sim_dsl.sh                                 |  45 +-
 test/dsl-st/cube_matrix_pipeline.py                | 113 ++---
 test/dsl-st/gemv_mx_pipeline.py                    |  16 -
 test/dsl-st/predicate_pack.py                      |  51 +--
 .../fold_tile_buf_intrinsics_dead_view_cleanup.pto |  45 ++
 test/tilelib-st/README.md                          | 361 ++++++++++++++++
 test/tilelib-st/a5/__main__.py                     |  32 ++
 test/tilelib-st/a5/tadd/case.py                    | 102 +++++
 test/tilelib-st/a5/tcolexpand/case.py              |  78 ++++
 test/tilelib-st/a5/tcolsum/case.py                 |  77 ++++
 test/tilelib-st/a5/tload_store/case.py             | 129 ++++++
 test/tilelib-st/a5/tmatmul/case.py                 | 203 +++++++++
 test/tilelib-st/common.py                          | 248 +++++++++++
 test/tilelib-st/run_tilelib_st.py                  |  36 ++
 tools/ptoas/driver.cpp                             |  36 +-
 29 files changed, 2227 insertions(+), 234 deletions(-)
===== END STAGE clone rc=0 @ 2026-07-06 16:20:34 =====

===== STAGE codex-review @ 2026-07-06 16:20:34 =====
set -euo pipefail
cd '/tmp/ptoas-pr-review-monitor/runs/20260706_162027_pr886/repo'
'codex' exec -C '/tmp/ptoas-pr-review-monitor/runs/20260706_162027_pr886/repo' -s read-only -c 'model_provider="codereview"' -c 'model="gpt-5.4"' -c 'model_reasoning_effort="xhigh"' --output-schema '/tmp/ptoas-pr-review-monitor/runs/20260706_162027_pr886/review_schema.json' -o '/tmp/ptoas-pr-review-monitor/runs/20260706_162027_pr886/codex_last_message.json' --color never - < '/tmp/ptoas-pr-review-monitor/runs/20260706_162027_pr886/review_prompt.txt'
[monitor] stage timeout: 1800s
OpenAI Codex v0.115.0 (research preview)
--------
workdir: /tmp/ptoas-pr-review-monitor/runs/20260706_162027_pr886/repo
model: gpt-5.4
provider: codereview
approval: never
sandbox: read-only
reasoning effort: xhigh
reasoning summaries: none
session id: 019f3683-ce3f-76e2-a50e-861939f93889
--------
user
你现在在审查 GitHub PR。

仓库:hw-native-sys/PTOAS
PR:#886 Add PTODSL A5 DSL ST coverage
作者:jimmychou0
base branch:origin/main
head branch:HEAD(当前已 checkout 到 PR head)

要求:
1. 只审查这个 PR 相对 origin/main 的改动,必要时可以看上下文文件。
2. 重点找真实的 correctness / regression / contract mismatch / CI / runtime / compatibility 问题。
3. 不要提纯风格建议,不要提低价值猜测。
4. 严格按优先级输出:
   - P1:高概率会导致错误结果、编译/运行失败、严重回归、发布阻断
   - P2:重要缺陷、行为回归、遗漏校验/测试、较大兼容性问题
   - P3:次要但明确可改的问题
5. 如果没有问题,summary 直接写:未检查到 PR #886 存在问题,并返回 findings=[]。
6. 如果有问题,summary 简洁概括,findings 里每条都要给出:
   - severity
   - title
   - body(说明为什么是问题,尽量具体)
   - file(尽量给相对路径)
   - line(能确定就填整数,否则 null)

建议先查看:
- git status --short
- git diff --stat origin/main...HEAD
- git diff --unified=80 origin/main...HEAD

最终输出必须严格匹配 JSON schema。

mcp startup: no servers
Reconnecting... 1/5 (unexpected status 403 Forbidden: {"code":"INSUFFICIENT_BALANCE","message":"Insufficient account balance"}, url: https://codex.0u0o.com/responses, cf-ray: a16d30eb7b2af317-LAS, request id: e58912ab-471e-4f62-92f1-835f6cc62acd)
Reconnecting... 2/5 (unexpected status 403 Forbidden: {"code":"INSUFFICIENT_BALANCE","message":"Insufficient account balance"}, url: https://codex.0u0o.com/responses, cf-ray: a16d30ee581b46c7-LAX, request id: a75eef12-ea7d-4bed-aa80-fb205a3ec3e8)
Reconnecting... 3/5 (unexpected status 403 Forbidden: {"code":"INSUFFICIENT_BALANCE","message":"Insufficient account balance"}, url: https://codex.0u0o.com/responses, cf-ray: a16d30f2fac6f4f2-LAS, request id: 0033c56d-2af9-4196-9847-e7b270d49672)
Reconnecting... 4/5 (unexpected status 403 Forbidden: {"code":"INSUFFICIENT_BALANCE","message":"Insufficient account balance"}, url: https://codex.0u0o.com/responses, cf-ray: a16d30f94d4572e0-LAX, request id: b0af09c0-ceba-4f0d-a555-aed62a7781c3)
Reconnecting... 5/5 (unexpected status 403 Forbidden: {"code":"INSUFFICIENT_BALANCE","message":"Insufficient account balance"}, url: https://codex.0u0o.com/responses, cf-ray: a16d31056eee1e0f-LAX, request id: 4f9a293d-55cd-43be-bb1b-cbcb1992edef)
ERROR: unexpected status 403 Forbidden: {"code":"INSUFFICIENT_BALANCE","message":"Insufficient account balance"}, url: https://codex.0u0o.com/responses, cf-ray: a16d311b8c1a1ddd-LAX, request id: 52ed954a-1a00-413b-84e7-504179da2e09
Warning: no last agent message; wrote empty content to /tmp/ptoas-pr-review-monitor/runs/20260706_162027_pr886/codex_last_message.json
===== END STAGE codex-review rc=1 @ 2026-07-06 16:20:43 =====

@jimmychou0 jimmychou0 force-pushed the codex/ptodsl-a5-dsl-st-validation branch 9 times, most recently from 9f6aa25 to 93ec308 Compare July 3, 2026 04:20
@jimmychou0 jimmychou0 marked this pull request as ready for review July 3, 2026 06:21
Comment thread test/dsl-st/cube_matrix_pipeline.py
Comment thread test/dsl-st/npu_a5/tadd.py Outdated
Comment thread lib/PTO/Transforms/FoldTileBufIntrinsics.cpp
Comment thread test/vpto/scripts/run_host_vpto_validation.sh Outdated
Comment thread tools/ptoas/driver.cpp
@jimmychou0 jimmychou0 force-pushed the codex/ptodsl-a5-dsl-st-validation branch 2 times, most recently from 4b25c38 to 26da009 Compare July 3, 2026 14:59
@jimmychou0 jimmychou0 force-pushed the codex/ptodsl-a5-dsl-st-validation branch from 26da009 to 8aa2570 Compare July 6, 2026 01:39
Comment thread .github/workflows/ci_sim.yml
PTOAS_BIN="${PTOAS_BIN:-${ROOT_DIR}/build/tools/ptoas/ptoas}"
PTOAS_FLAGS="${PTOAS_FLAGS:---pto-arch a5 --pto-backend=vpto}"
# set he HOST_RUNNER to "ssh root@localhost" if must change user to root to access the device
HOST_RUNNER="${HOST_RUNNER:-}"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

没懂为什么要修改vpto validation相关的2个脚本,这个PR并没有改任何vpto的st用例

Comment thread test/tilelib-st/a5/tadd/case.py
Comment thread test/dsl-st/npu_a5/tadd.py Outdated

A_TILE_ADDR = 0
B_TILE_ADDR = 4096
C_TILE_ADDR = 8192

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PTOAS支持自动分配tile地址,作为打样的话可以引导后续开发者往这个方向写,应该能简化很多东西

@jimmychou0 jimmychou0 force-pushed the codex/ptodsl-a5-dsl-st-validation branch 3 times, most recently from d3bf2ce to e114140 Compare July 6, 2026 07:09
@jimmychou0 jimmychou0 force-pushed the codex/ptodsl-a5-dsl-st-validation branch from e114140 to 445f437 Compare July 6, 2026 07:49
@jimmychou0 jimmychou0 force-pushed the codex/ptodsl-a5-dsl-st-validation branch from 445f437 to c35bac0 Compare July 6, 2026 08:18
@zhangstevenunity zhangstevenunity merged commit 4b0ace6 into hw-native-sys:main Jul 6, 2026
14 of 16 checks passed
@reedhecre

Copy link
Copy Markdown

A3 板测完成(有跳过)

  • 触发方式:merged
  • 源码提交:4b0ace6cab87
  • 结果汇总:OK 220 / FAIL 0 / SKIP 2
  • 日志:/home/zhongxuan/ptoas-board-monitor/runtime/logs/20260706_175832_merged_pr886.log
  • LLVM cache:/home/zhongxuan/ptoas-board-monitor/cache/llvm-project-vpto-llvm21/build-shared
  • 结果 TSV:/home/zhongxuan/ptoas-board-monitor/runtime/logs/20260706_175832_merged_pr886.tsv

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants